Fisher information and Hamilton's canonical equations 

F. Pennini and A. Plastino 
Instituto de Fisica La Plata (IFLP) 
Universidad Nacional de La Plata (UNLP) and 
Argentine National Research Council ( CONICET) 
C.C. 727, 1900 La Plata, Argentina 

Abstract 

We show that the mathematical form of the information measure of Fisher's I for a Gibbs' 
canonical probability distribution (the most important one in statistical mechanics) incorporates 
important features of the intrinsic structure of classical mechanics and has a universal form in 
terms of "forces" and "accelerations", i.e., one that is valid for all Hamiltonian of the form T + V. 
If the system of differential equations associated to Hamilton's canonical equations of motion is 
linear, one can easily ascertain that the Fisher information per degree of freedom is proportional 
to the inverse temperature and to the number of these degrees. This equipartition of / is also seen 
to hold in a simple example involving a non-linear system of differential equations. 

PACS numbers: 02.50.-r, 05.20.-y, 45.20.-d 
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INTRODUCTION 



The last years have witnessed a great deal of activity revolving around physical applications 



of Fisher's information measure (FIM) (as a rather small sample, see for instance, 

n n 

|5j|). Frieden and Soffer [1] have shown that Fisher's information measure provides one with a 
powerful variational principle, the extreme physical information one, that yields most of the 
canonical Lagrangians of theoretical physics Additionally, I has been shown to provide 

an interesting characterization of the "arrow of time" , alternative to the one associated with 
Bolton, ea,o Py QQ. Also to be atentioned is a recent approach to aon-e^riun, 
thermodynamics, based upon Fisher's measure (a kind of "Fisher-MaxEnt"), that exhibits 
definite advantages over conventional text-book treatments js, |^|. Thus, unravelling the 
multiple FIM facets and their links to physics should be of general interest to a vast audience. 
R. A. Fisher advanced, already in the twenties, a quite interesting information measure (for 
a detailed study see Q, Consider a 9 — z "scenario" in which we deal with a system 
specified by a physical parameter 9, while z is a stochastic variable (z G 3? M ) and fg(z) 
the probability density for z (that depends also on 9). One makes a measurement of z 
and has to best infer 9 from this measurement, calling the resul ting estimate 9 = 6(z). 
The question is how well 9 can be determined. Estimation theory [UJ states that the best 
possible estimator 6(z), after a very large number of z-samples is examined, suffers a mean- 
square error e 2 from 9 that obeys a relationship involving Fisher's J, namely, Ie 2 = 1, where 
the Fisher information measure / is of the form 

/(«) = /dz/ 8 (z){^)} 2 . (1) 

This "best" estimator is the so-called efficient estimator. Any other estimator exhibits a 
larger mean-square error. The only caveat to the above result is that all estimators be 
unbiased, i.e., satisfy (9(z)) = 9. Fisher's information measure has a lower bound: no 
matter what parameter of the system one chooses to measure, I has to be larger or equal 
than the inverse of the mean-square error associated with the concomitant experiment. 
This result, I e 2 > 1, is referred to as the Cramer-Rao bound [2J|. A particular J-case is of 

hn 

great importance: that of translation families [2j, |3(, i.e., distribution functions (DF) whose 
form does not change under ^-displacements. These DF are shift-invariant (a la Mach, 
no absolute origin for 9), and for them Fisher's information measure adopts the somewhat 



2 



simpler appearance 
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/I d In /(z) I 
dz/(z) < — — > . (2) 

Fisher's measure is additive Q]. If one is dealing with phase-space, z is a M = 2N- 
dimensional vector, with N "coordinates" r and N "momenta" p. As a consequence, if 
one wishes to estimate phase-space "location" , one confronts an J(r + p) = /(z)-measure 
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and 



J(z)=/(r) + /(p). (3) 



Here we clearly encounter a shift-invariance situation. 



HAMILTONIAN SYSTEMS 

Assume we wish to describe a classical system of N identical particles of mass m whose 
Hamiltonian is of the form (in self-explanatory notation) 

H = T + V. (4) 

A special important instance is that of 

N 2 N 

W('.P)=E5: + E^ i ) (5) 
i=i z '' n i=i 

where (r,, p«) are the coordinates and momenta of the i-th particle and V(rj) is a central 
single-particle potential. Our considerations are not limited to Hamiltonians (0), though. 
We presuppose that the system is in equilibrium at temperature T, so that, in the canonical 
ensemble, the probability density reads 

e -PH(T,p) 

P(r,p) = ^ (6) 

f h 



with k Boltzmann's constant and r — p denoting two 3N— dimensional vectors, 
denotes an elementary cell in phase-space, we write, with some abuse of notation 
dr = d 3N r d 3N p/(N\ h 3N ) for the pertinent integration-measure while the partition function 



reads 
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}3N r ^3N 



J N\ h 3N ' 1 } 
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Remember now Hamilton's celebrated canonical equations 



so that, obviously, 



&H(r,p) 
dp 



dH(r,p) 
dr 



P, 



(8) 



kT d\np{r,p) = . 



—kT 



dp 
din p(r, p) 

dr 



P 



(9) 



Eqs. (jnj) are just the above referred to canonical equations of Hamilton in a different guise. 
We recall them here because the expressions in the left-hand-sides of (JHJ) enter the definition 
of Fisher's measure, if one expresses I T as a functional of the probability density p. Indeed, 
this FIM has the form (J2J), so that, in view of Q, it acquires the aspect 

d 37V rd 37V p 



N\ h 3N 



p(r,p)A(T,p) 



with 



A = a 



din p(r, p) 
dp 



+ b 



where, for the sake of dimensional balance 



d In p(r, p) 
dr 



(10) 



(11) 



ll| . one needs two coefficients a and b that 



depend on the specific Hamiltonian of the system under scrutiny (see the following Section). 
Notice that 



and 



d In p(r, p) 
dp 

din p(r, p) 
dr 



dp 

dH(r,p) 
dr 



so that we can rewrite the Fisher information as 

2\ 



(kT) 2 I T = a 



fdH(p, 



'dH(r,p) 



(12) 



(13) 



(14) 



y dp ) I \\ dr ) I 
In view of Eqs. (JUJ), Fisher's measure now becomes 

J r = /3 2 {a (r 2 ) + 6(p 2 )}. 

We see that it incorporates the symplectic structure of classical mechanics. Eqs. J77|I73)) are 
our main original results here: they give the FIM- form for any Hamiltonian of the form (jljl. 



(15) 
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a universal "quadratic" form in the average values of "square-accelerations" and "square- 
forces" . The first term in the right-hand-side of (fPfj) is in fact the same for all systems for 
which T is the conventional kinetic energy. Note that cyclic canonical coordinates do not 
contribute to I T . 

SOME SPECIALLY IMPORTANT EXAMPLES: GENERAL QUADRATIC HAMIL- 
TONIANS 

In many important cases one faces Hamiltonians of the type 

N 

H L = J2(A i P? + B i Q 2 ), (16) 

8=1 

with Pi — Qi generalized momenta and coordinates, respectively, that lead to 2N differential 
equations, the so-called linear Hamiltonian problem [13j]. According to the preceding section 
we have now 



Aj = \/(/3oAi) and E>i = l/((3 Bi) for all i, where (3q is a fixed, arbitrary reference inverse- 
temperature. Further reflection allows one to see that 



We know that the equipartition theorem holds for {Hl) [12], so that the mean value of the 
energy equals 2NkT, which entails that it also holds for the Fisher information 



As expected, the information steadily diminishes as the temperature grows. The divergence 
at zero temperature is of no importance, since classical mechanics is not valid at low enough 
temperatures. 

Free particle 




(kT) 2 I T = (H L )/p . 



(18) 



I T = 2N{/3//3 ). 



(19) 
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In the case of the N identical free particles, the Hamiltonian is TCj = p 2 /2m with i = 1 . . . N. 
We make here the obvious choice a = m//3o (mass). Since (r 2 ) = 2(H) /m we have 



I T = 2(3 2 (H/Po) 



(20) 



so that, using the well-known result (equipartition) 12j (H) = 3N/2f3 one obtains 



J T = 3iV (p/fo) 



(21) 



N harmonic oscillators 

We consider N identical harmonic oscillators whose Hamiltonian is Hi = p 2 /2m + moor 2 /2 
with % = 1 . . . N. It is easy to calculate the classical mean values in (|15p. which have the 
form (r 2 ) = 3N/mP and (p 2 ) = 3Nmoo 2 / p. In this case the proper choice is a = m/Po and 
b = l/mP Q uj 2 , so that the information (|15|) adopts the appearance 

I T = 6N(P/p ). (22) 

In these two examples we appreciate again the fact that a Fisher-equipartition emerges, 
kT / p per degree of freedom. 



PARAMAGNETIC SYSTEM 



Consider N magnetic dipoles, each of magnetic moment /x, in the presence of an external 
magnetic field H of intensity H. These N distinguishable (localized), identical, mutually 
non-interacting, and freely orientable dipoles give rise to a dipole-potential energy 

N N 

W = -£ JIi.-n = -nH cos 6 h (23) 

i=l i=l 

where 9i gives the dipole orientation with respect to the field-direction. Since there is no 
interaction between our iV particles, and both I and the entropy S are additive quantities, 
it is permissible to focus attention on just one generic particle (whose canonical conjugate 
variables we call (9,p$)) and, at the end of the calculation, multiply by N. Hamilton's 
canonical equations yield then the non-linear equation 



Po = fiH sin 9. 
6 



(24) 



Notice that po does not appear in TC, i.e., dH/dpe = 9 = 0, which entails, of course, (9) = 0. 
Thus, in choosing the constants entering the definition of I T (see the preceding Section), we 
need only to care about b, with b = l/(fi/3 H). The associated Fisher information is then 
(Cf. Eq. JED; 

R' 2 AT , 

(25) 



I T = — — ( sin 9 ) . 



We can easily compute the above mean value using (sin9 d9 dip) for the elemental solid angle, 
so that 

(sin 2 9) = -\ r r e^ H sin 3 9 d9 dtp (26) 
Z Jo Jo 



where the partition function per particle is of the form 

sinh(a;) 
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Z = 4tT' 



x 



(27) 



with x = [iH/kT. Evaluating the integral ()26|) explicitly we obtain 

I T = 2N(J3//3o)L(x), 

an original result, where L(x) is the well known Langevin function 

1 



(28) 
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L(x) = coth x — 



x 



(29) 



Since L(x) vanishes at the origin, so does I T for infinite temperature, a result one should 
expect on more general grounds d|. Again, equipartition of Fisher information ensues. 
Since the differential equations are not linear in the conjugate (canonical) variables, the 
information per degree of freedom does not equal that of (fT9~|) . 



CONCLUSIONS 



We summarize now our results. In this communication we have shown that 

• the mathematical form of the information measure of Fisher's / for a Gibbs' canonical 
probability distribution incorporates important features of the intrinsic structure of 
classical mechanics 

• it has a universal form in terms of "forces" and "accelerations", i.e., one that is valid 
for any Hamiltonian of the form T + V 
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• if the system of differential equations associated to Hamilton's canonical equations of 
motion is linear, the amount of Fisher information per degree of freedom is propor- 
tional to the inverse temperature and to the number of these degrees (equipartition of 
information!) 

• equipartition of / has been seen to hold also for paramagnetic systems, for which a 
non-linear system of differential equations is involved. 




FIG. 1: We plot i) the entropy S and ii) the Fisher information I T per particle vs. x = fiH/kT for 
iV magnetic dipoles, each of magnetic moment n, in the presence of an external magnetic field H 
of intensity H. 
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